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Over  Che  past  two  decades,  a  number  of  strategies  used  by  people  to 
evaluate  decision  alternatives  have  been  identified  (Svenson,  1979;  Abelson 
and  Levi,  1985).  Some  of  the  strategies  involve  the  consideration  of  all  the 
relevant  decision  information  in  a  normative  fashion.  For  example,  the 
Weighted  Additive  (WADD)  rule  considers  the  values  of  each  alternative  on  all 
the  relevant  attributes  and  considers  all  the  relative  importances  (weights  or 
probabilities)  of  the  attributes  (outcomes)  to  the  decision  maker  in  a 
compensatory  fashion.  Other  strategies  identified  involve  the  selective 
processing  of  much  less  information  and  are  noncompensatory.  For  example,  the 
lexicographic  choice  rule  (Tversky,  1969)  simply  chooses  the  alternative  which 
is  best  on  the  most  important  attribute,  ignoring  much  of  the  potentially 
relevant  problem  information. 

A  second  major  finding  from  years  of  decision  research  is  that  the  same 
individual  will  often  employ  diverse  information  processing  strategies  in 
making  a  decision,  contingent  on  task  demands  (Einhorn  and  Hogarth,  1981; 
Payne,  1982;  Hammond,  1986).  For  example,  when  faced  with  a  decision  problem 
involving  only  two  or  three  alternatives,  people  generally  use  more 
compensatory  types  of  strategies.  When  faced  with  more  complex 
(multialternative)  decision  tasks,  people  tend  to  use  noncompensatory 
strategies  such  as  elimination-by-aspects  (EBA)  (Tversky,  1972). 

Given  the  substantial  evidence  for  contingent  decision  behavior,  a 
natural  question  is  why  decision  makers,  given  a  particular  task,  select  one 
particular  decision  strategy  instead  of  another.  The  .nose  frequently 
advocated  approach  to  answering  that  question  is  to  assume  that  the  selection 
among  decision  strategies  is,  in  part,  a  function  of  the  strategy's  accuracy 
and  the  strategy's  effort  (i.e.,  its  demand  for  mental  resources)  (Beach  and 
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Mitchell,  1978;  Russo  and  Dosher,  1983;  Johnson  and  Payne,  1985).  The  effort- 
accuracy  framework  provides  a  way  of  explaining  contingent  decision  behavior, 
since  the  effort  and  accuracy  associated  with  various  strategies  are  likely  tc 
vary  across  problems. 

Recently,  Payne,  Bettman,  and  Johnson  (1988)  examined  the  role  of  effort 
and  accuracy  considerations  in  the  adaptive  use  of  decision  strategies.  They 
were  concerned  with  the  extent  to  which  a  decision  maker  chooses  strategies 
that  are  relatively  efficient  in  terms  of  effort  and  accuracy  as  task  and 
context  demands  are  varied.  Monte-Carlo  simulation  was  used  to  examine  the 
impact  of  variation  in  problem  size,  the  presence  or  absence  of  time  pressure, 
the  presence  or  absence  of  dominated  alternatives,  and  dispersion  in 
probabilities  or  attribute  weights^"  on  the  accuracy  and  effort  of  a  variety  of 
choice  heuristics.  Choice  heuristics  which  approximated  the  accuracy  of 
normative  procedures  while  requiring  substantially  less  effort  were 
identified,  indicating  that  heuristics  can  provide  an  efficient  form  of 
decision  processing  (Thorngate,  1980) .  However,  no  single  heuristic  did  well 
across  all  the  task  and  context  conditions  studied  by  Payne  et.  al.  (1988). 

The  implication  is  that  a  decision  maker  striving  to  maintain  a  high  level  of 
accuracy  with  a  minimum  of  effort  would  have  to  adaptively  use  a  variety  of 
heuristics . 

Payne  et.  al.  (1988)  next  investigated  whether  the  actual  decision 
behavior  exhibited  by  subjects  was  consistent  with  the  processing  patterns 
identified  by  the  Monte-Carlo  simulations.  The  results  provided  strong 
evidence  of  adaptivity  in  decision  making.  Generally,  the  subjects  changed 
processing  strategies  in  ways  that  the  simulation  indicated  were  appropriate 
given  changes  in  the  structure  of  the  decision  tasks.  For  example,  when  the 
dispersion  in  probabilities  (weights)  was  high  rather  than  low,  subjects 
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generally  acquired  less  information,  spent  less  time  per  acquisition,  were 
more  selective  regarding  which  information  was  examined,  and  processed 
information  more  by  attribute  than  by  alternative.  Payne  et.  al .  suggested 
that  the  shifts  in  processing  reflected  a  shift  in  strategies  from  a 
compensatory  type  strategy  in  the  low  dispersion  condition  to  a  lexicographic 
or  EBA  type  of  strategy  in  the  high  dispersion  condition. 

Although  clear  evidence  of  adaptivity  was  obtained,  several  unresolved 
issues  regarding  such  adaptive  selection  remain.  For  instance,  the  evidence 
reported  in  Payne,  Bettman,  and  Johnson  (1988)  suggests  that  people  were 
learning  to  adapt  their  decision  strategies  to  changes  in  task  and  context. 

Yet  in  none  of  their  experiments  did  Payne  et.  al .  provide  their  subjects  with 
explicit  accuracy  or  outcome  feedback.  In  such  situations,  how  might  an 
individual  learn  when  and  how  to  change  decision  strategies?  In  addition,  if 
individuals  make  effort  -  accuracy  tradeoffs  in  strategy  selection,  then  the 
strategies  used  should  vary  if  the  relative  weight  placed  on  effort  versus 
accuracy  is  changed.  Payne  et.  al .  did  not  examine  this  aspect  of  adaptation. 
These  two  major  issues  are  examined  in  the  present  study:  1)  How  decision 
makers  learn  to  adapt,  and  2)  How  the  relative  weight  placed  on  effort  versus 
accuracy  affects  decision  processes. 

Learning  to  Adapt 

For  an  individual  to  adapt  decision  strategies  to  a  particular  decision 
task,  he  or  she  must  have  at  least  some  vague  ideas  about  the  degree  of  effort 
and  accuracy  characterizing  his  or  her  decision  process.  Payne  et.  al . 
speculate  that  process  feedback  (Anzai  and  Simon,  1979)  could  easily  provide 
information  on  the  effort  involved  in  using  a  strategy.  Process  feedback  is 
information  about  the  course  of  one's  own  decision  processes.  In  the  course  of 
solving  a  decision  problem,  an  individual  probably  has  a  fairly  rich  data  base 
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available  about  how  effortful  and/or  difficult  he  or  she  is  finding  the 
decision.  In  contrast,  gaining  information  on  accuracy  from  process  feedback 
in  these  experiments  would  seem  more  problematic.  One  does  not  know  exactly 
whether  the  final  decision  is  a  good  one  or  not. 

A  self -generated  notion  of  accuracy  is  possible  to  some  degree  if  we 
assume  that  individuals  have  general  knowledge  about  the  properties  of  a 
reasonable  strategy,  at  least  for  some  decision  tasks.  For  example,  decision 
makers  might  believe  that  a  "good"  strategy  involves  first  looking  at  the  most 
important  information  for  all  alternatives,  and  then  looking  at  other 
information  as  desired  or  as  time  allows  (Payne,  Bettman,  and  Johnson  (1988) 
report  data  supporting  this  notion) .  An  individual  could  Chen  observe  how 
well  his  or  her  choice  process  as  executed  matched  this  general  notion  of  a 
"good"  strategy.  For  example,  if  time  ran  out  before  some  important 
information  could  be  considered,  then  the  individual  might  decide  that  the 
process  should  be  changed.  Such  use  of  process  feedback  to  assess  accuracy  is 
similar  to  ideas  of  Reder  (1987),  who  suggests  that  people  generate  "feelings 
of  knowing"  about  the  quality  of  their  performance  during  the  course  of 
problem  solving. 

The  present  study  examines  how  decision  makers  may  learn  to  adapt  when 
presented  with  explicit  accuracy  feedback  and/or  explicit  effort  feedback. 

The  notions  about  process  feedback  outlined  above  suggest  some  interesting 
hypotheses  in  this  situation.  For  example,  the  presence  of  explicit  accuracy 
feedback  should  only  improve  the  quality  of  decisions  for  problems  where 
individuals  do  not  have  strong  notions  of  a  "good"  strategy.  Thus,  the 
effects  of  accuracy  feedback  should  vary  across  decision  tasks.  For  instance, 
the  general  notion  of  a  "good"  strategy  outlined  above  (i.e.,  look  at  the  most 
important  information  first)  is  less  easy  to  execute  when  faced  with  problems 


5 


involving  low  dispersion,  since  one  cannot  easily  be  selective.  Payne, 

Bettman,  and  Johnson  (1988)  also  found  that  low  dispersion  problems  took 

longer  to  solve.  In  addition,  subjects  report  when  asked  that  decisions  with 

low  dispersion  are  more  difficult  and  that  they  have  less  confidence  in  their 

2  * 

decisions  when  dispersion  is  low.  Given  these  perceptions,  process  feedback 
seems  less  likely  to  suffice.  Hence,  we  predict  that  explicit  accuracy 
feedback  will  have  greater  effects  on  low  dispersion  decision  problems. 

The  discussion  of  process  feedback  above  implies  that  individuals  are 
able  to  generate  fairly  accurate  notions  about  their  own  effort  levels. 
Therefore,  the  addition  of  explicit  effort  feedback  should  provide  little 
additional  information  which  might  affect  individuals'  decision  processes. 
While  effort  feedback  may  not  provide  any  additional  information  to  the 
decision  maker,  both  effort  and  accuracy  may  exert  motivational  influences  on 
the  choice  process.  Such  influences  are  considered  below.  It  should  be 
noted,  however,  that  although  we  have  separated  the  information  provided  by 
feedback  from  its  motivational  effects  in  this  discussion,  informational  and 
motivational  effects  are  extremely  difficult  to  distinguish,  both  empirically 
and  theoretically  (Tetlock  and  Levi,  1982).  We  use  the  terms  motivational  and 
informational  for  convenience  in  describing  our  thinking  and  hypotheses; 
however,  our  focus  is  on  the  overall  pattern  of  results  we  obtain,  not  on 
distinguishing  these  two  types  of  effects  empirically. 

Weighting  Effort  versus  Accuracy 

When  deciding  how  to  decide,  the  effort-accuracy  framework  assumes  that 
a  decision  maker  evaluates  the  costs  (i.e.,  effort)  and  benefits  (i.e., 
accuracy)  of  the  various  strategies  that  are  known  to  him  or  her  and  selects 
that  strategy  which  is  in  some  sense  best  for  the  environment.  Such  a 
framework  implies  that  strategy  selection  will  be  guided  by  the  relative 
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salience  of  effort  and  accuracy  in  the  decision  environment.  A  second  purpose 
of  the  present  study  is  to  directly  examine  how  decision  processing  varies  as 
the  goals  and  incentives  in  a  decision  environment  are  changed  to  shift  the 
relative  emphasis  on  accuracy  and  effort.  We  hypothesize  that  a  goal 
emphasizing  accuracy  to  a  greater  extent  will  lead  to  more  normative  types  of 
processing  and  improved  accuracy.  A  goal  emphasizing  effort  minimization  to  a 
greater  extent  is  expected  to  lead  to  more  use  of  heuristics  like  elimination- 
by-aspects  or  the  lexicographic  rule. 

As  noted  above,  explicit  accuracy  feedback  and  explicit  effort  feedback 
could  have  motivational  implications,  in  addition  to  providing  information 
about  the  efficiency  of  decision  strategies.  For  example,  the  availability  of 
explicit  effort  feedback  might  increase  the  saliency  of  the  effort  expended 
during  the  choice  process.  Similarly,  explicit  accuracy  feedback  could 
enhance  the  concern  with  accuracy.  We  hypothesize  that  these  motivational 
effects  of  feedback  will  interact  with  the  goal  of  the  decision  maker.  In 
particular,  making  effort  more  salient  by  providing  effort  feedback  might  lead 
to  more  heuristic  processing  and  poorer  performance  when  the  goal  is  to 
minimize  effort;  however,  when  the  goal  is  to  maximize  accuracy,  highlighting 
effort  via  effort  feedback  might  make  the  decision  maker  work  harder  and  could 
improve  performance.  Subjects  often  believe  that  greater  effort  does  lead  to 
better  performance  (Yates  &  Kulick,  1977).  Likewise,  accuracy  feedback  may 
interact  with  the  decision  maker's  goal.  If  the  goal  is  to  maximize  accuracy, 
the  decision  maker  may  use  accuracy  feedback  to  the  best  of  his  or  her  ability 
in  order  to  try  to  process  more  normatively  and  improve  performance;  however, 
providing  accuracy  feedback  may  have  little  effect  on  the  decision  maker  if 
the  goal  is  to  minimize  effort. 
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To  summarize,  the  purposes  of  this  paper  are  to  1)  examine  the  impact  of 
feedback  on  accuracy  and  effort  on  the  adaptive  use  of  decision 
strategies;  and  2)  to  investigate  how  changes  in  goals  influence  the  use 
of  various  decision  strategies.  We  hypothesize  the  following: 

1)  Accuracy  feedback  will  have  its  greatest  impact  when  decision  makers 
are  faced  with  more  difficult  problems,  such  as  those  where  the 
dispersion  in  weights  (probabilities)  is  low.  The  impact  of  accuracy 
feedback  will  be  to  increase  the  use  of  more  normative  decision 
strategies  like  the  weighted  additive  rule.  Such  strategies  are 
characterized  by  a  substantial  amount  of  processing;  an  amount  of 
processing  that  is  not  selective  but  rather  is  consistent  across 
alternatives  and  attributes,  indicating  more  compensatory  processes 
fPayne,  1976);  and  processing  that  is  alternative-based  rather  than 
attribute-based  (that  is,  multiple  items  of  information  about  a  single 
alternative  are  processed  before  any  information  about  another 
alternative  is  processed) . 

2)  The  availability  of  effort  feedback  will  have  little  direct  impact 
on  decision  behavior. 

3)  A  goal  statement  that  emphasizes  accuracy  more  than  effort  will  lead 
to  the  use  of  more  normative  strategies  like  WADD.  In  contrast,  a  goal 
statement  that  emphasizes  effort  savings  more  than  accuracy  will  lead  to 
the  use  of  strategies  like  the  lexicographic  rule  or  elimination-by- 
aspects.  Such  strategies  are  characterized  by  a  smaller  amount  of 
processing,  processing  that  is  selective  across  attributes  and/or 
alternatives,  and  processing  that  is  attribute-based.  This  last 
prediction  is  based  on  the  suggestion  by  Tversky  (1969)  and  by  Russo  and 
Dosher  (1983)  that  attribute -based  processing  is  cognitively  easier. 
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4)  Effort  and  accuracy  feedback  will  interact  with  the  goal  of  the 
decision  maker  to  jointly  determine  performance  and  type  of  processing. 
Both  effort  and  accuracy  feedback  will  lead  to  more  normative  processing 
and  better  performance  when  the  goal  is  to  maximize  accuracy.  When  the 
goal  is  to  minimize  effort,  however,  effort  feedback  will  foster  more 
heuristic  processing  and  decreased  performance,  while  accuracy  feedback 
will  have  little  effect. 

Method 

Overview 

Subjects  used  a  computer-based  information  acquisition  system  (Johnson, 
Payne,  Schkade ,  and  Bettman,  1986)  to  acquire  information  and  make  decisions 
among  32  sets  of  multiattribute  nonrisky  alternatives.  The  subjects'  task  was 
to  select  the  alternative  in  each  set  that  they  thought  was  best  overall. 

Each  set  contained  four  alternatives  defined  by  six  attributes.  The  sets 
varied  within  subjects  with  respect  to  1)  the  dispersion  of  the  weights 
provided  for  the  attributes  (high  or  low)  and  2)  the  explicit  goal  of  the 
decision  maker  for  the  set  (minimize  effort  or  maximize  accuracy) .  The 
between- subj ect  factors  were  the  presence  or  absence  of  1)  effort  feedback  and 
2)  accuracy  feedback. 

Subjects 

Eighty-one  undergraduates  at  a  large,  northeastern  university  enrolled 
in  an  introductory  marketing  course  served  as  subjects.  Participation  in  the 
experiment  earned  credit  towards  fulfillment  of  a  course  requirement.  In 
order  to  encourage  the  subjects  to  do  their  best,  they  were  told  that  they 
were  eligible  to  enter  a  $100  lottery  if  their  performance  exceeded  a 
standard.  Subjects  were  told  that  an  index  of  overall  performance  would  be 
developed  based  upon  the  time  taken  and  the  accuracy  achieved  for  each  trial. 
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Subjects  were  instructed  that  for  trials  when  the  goal  was  to  minimize  effort, 
time  taken  would  be  given  a  weight  of  three  and  accuracy  a  weight  of  one.  On 
trials  where  the  goal  was  to  maximize  accuracy,  time  taken  would  receive  a 
weight  of  one  and  accuracy  a  weight  of  three.  Thus,  both  accuracy  and  effort 
(time  taken)  mattered  for  all  trial;  we  tried  to  manipulate  the  relative 
importance  of  those  two  issues.  However,  in  all  cases  choosing  an  accurate 
response  quickly  would  lead  to  a  higher  index  of  overall  performance. 

Subjects  were  told  that  these  scores  would  be  aggregated  for  all  32  trials, 
and  that  subjects  whose  index  met  a  minimum  requirement  (which  was  not  further 
specified)  would  be  eligible  for  the  lottery.  In  fact,  all  subjects  were 
entered  into  the  lottery. 

Stimuli 

The  stimuli  were  sets  of  four  nonrisky  options  with  six  attributes.  The 
values  of  the  attributes  were  single  digits  ranging  from  2  (least  preferred) 
to  9  (most  preferred) .  A  set  of  weights  for  the  attributes  was  also 
presented,  with  the  weights  constrained  to  sum  to  100.  The  alternatives  were 
simply  labeled  Alternative  A  through  Alternative  D,  with  attributes  labeled 
simply  Attributes  1  through  Attribute  6. 

Eight  sets  of  lew  dispersion  in  weights  options  and  eight  sets  of  high 
dispersion  options  were  generated,  with  the  possibility  of  dominated  options 
allowed  in  all  sets.  To  illustrate,  one  low  dispersion  of  alternatives 

had  weights  of  13,  20,  13,  28,  12,  and  14  for  the  six  attributes.  One  high 
dispersion  set  of  options  nad  weights  of  70,  12,  3,  4,  3,  and  6. 

Each  of  these  16  sets  of  options  was  presented  to  subjects  twice  in 
order  to  manipulate  the  subject's  goal  for  the  decision.  For  one 
presentation,  the  subject  was  instructed  that  minimization  of  the  effort  used 
to  decide  among  the  alternatives  was  relatively  more  important.  For  the  other 
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presentation,  subjects  were  told  that  the  maximization  of  the  accuracy  of 
their  choice  was  relatively  more  important.  Thus,  each  subject  received  32 
decision  problems  (2  dispersion  conditions  x  2  goal  conditions  x  8 
replications).  The  use  of  a  within- subjects  design  for  dispersion  and  goal 
provides  a  strong  test  of  adaptivity,  since  subjects  would  be  expected  to 
switch  strategies  from  one  decision  problem  to  the  next. 

The  first  16  problems  presented  to  each  subject  contained  four  sets 
corresponding  to  each  of  the  four  dispersion  x  goal  combinations,  as  did  the 
second  16  problems.  Within  each  block  of  16  problems,  the  problem  sets  were 
randomly  arranged,  with  the  same  random  order  for  each  subject.  A  complete 
experimental  session  took  from  40-60  minutes  for  each  subject. 

Information  Acquisition  Methodology 

Information  acquisitions,  response  times,  and  choices  were  monitored 
using  a  software  system  called  Mouselab  (Johnson,  Payne,  Schkade ,  &  Bettman, 
1986).  This  system  uses  an  IBM  personal  computer,  or  equivalent,  equipped 
with  a  "mouse"  for  moving  a  cursor  around  the  display  screen  of  the  computer. 
The  stimuli  are  presented  on  the  display  in  the  form  of  a  matrix  of  available 
information.  The  first  row  of  boxes  contained  information  about  the  weights 
for  the  six  attributes.  The  next  four  rows  of  boxes  contained  information 
about  the  values  associated  with  the  different  outcomes  for  each  alternative, 
respectively.  At  the  bottom  of  the  screen  .e four  boxes  that  were  used  to 
indicate  which  alternative  was  most  preferred. 

When  a  set  of  alternatives  first  appears  on  the  screen,  the  values  of 
the  attributes  and  weights  are  "hidden"  behind  the  labeled  boxes.  To  open  a 
particular  box  and  examine  the  information,  the  subject  has  to  move  the  cursor 
into  the  box.  The  box  immediately  opens  and  remains  open  until  the  cursor  is 
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moved  out  of  the  box.  Only  one  box  can  be  open  at  a  time.  Subjects  could 
acquire  as  much  or  as  little  information  as  they  desired. 

The  Mouselab  program  records  the  order  in  which  boxes  are  opened,  the 
amount  of  time  boxes  are  open,  the  chosen  option,  and  the  total  elapsed  time 
since  the  display  first  appeared  on  the  screen.  Response  times  are  recorded 
to  an  accuracy  of  l/60th  of  a  second. 

Feedback  Manipulations 

Accuracy  Feedback.  One  between-subjects  factor  was  the  presence  or 

absence  of  explicit  accuracy  feedback.  Before  defining  the  form  of  such 

feedback,  we  must  first  consider  how  to  define  subjects'  accuracy  levels. 

While  accuracy  of  choice  can  be  defined  in  many  ways  (e.g.,  not  selecting 

dominated  alternatives  or  avoiding  intransitive  patterns  of  preferences) ,  the 

compensatory  weighted  additive  rule  is  often  used  as  a  criterion  for  decision 

effectiveness  in  multiattribute  choice  (Zakay  &  Wooler,  1984).  The  weighted 

additive  rule  is  a  normative  procedure  in  that  all  information  is  processed 

and  tradeoffs  reflecting  the  decision  maker's  values  are  incorporated  into  the 

decision.  A  measure  of  the  accuracy  of  a  decision  can  be  derived  by 

comparing  the  weighted  additive  value  for  the  alternative  selected  from  a  set 

to  the  best  and  worst  possible  values  for  that  set: 

Weighted  Additive  Valuec^0^ce-Weighted  Additive  ValueWorst 

Relative  Accuracy  —  - 

Weighted  Additive  Valuegest  *  Weighted  Additive  Valueyors;_ 

This  measure  is  bounded  by  a  value  of  1  if  the  option  with  the  highest 

weighted  additive  value  is  selected  and  a  value  of  0  if  the  option  with  the 

worst  weighted  additive  value  is  chosen.  It  provides  a  measure  of  the 

relative  improvement  of  a  choice  over  the  worst  possible  choice.  For  further 

discussion  of  this  and  other  measures  of  accuracy,  see  Johnson  &  Payne  (1985) 

or  Payne,  Bettman,  &  Johnson  (1988). 
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This  measure  was  used  to  provide  accuracy  feedback  to  subjects  via 
Mouselab.  For  each  choice  set,  the  relative  accuracy  score  corresponding  to 
each  option  was  computed.  After  the  subject  had  made  a  choice  for  a  problem 
set,  a  scale  ranging  from  0  to  1  appeared  on  the  screen.  The  letters  A 
through  D  were  positioned  on  the  scale  to  correspond  to  their  relative 
accuracy  scores.  For  example,  if  the  weighted  additive  values  for 
alternatives  A  to  D  were  125,  200,  150,  and  100,  then  the  relative  accuracy 
scores  would  be  .25,  1,  .5,  and  0  respectively.  Alternative  D  would  be  placed 
above  the  left  end  of  the  scale,  Alternative  A  would  be  one-quarter  of  the  way 
toward  the  right  end  of  the  scale ,  Alternative  C  would  be  above  the  midpoint 
of  the  scale,  and.  Alternative  B  would  be  above  the  right  end  of  the  scale. 

Subjects  could  see  the  relative  positioning  of  the  alternatives  and  thus 
receive  visual  feedback.  In  addition,  subjects  could  move  the  mouse  along  the 
scale  to  correspond  to  the  letter  for  any  alternative.  The  exact  relative 
accuracy  score  for  that  alternative  would  then  be  displayed. 

Effort  Feedback.  The  second  between- subjects  factor  was  the  presence  or 
absence  of  explicit  effort  feedback.  For  subjects  in  the  effort  feedback 
condition,  a  clock  was  displayed  in  the  top  left  corner  of  the  screen  for  each 
set  of  alternatives  presented.  The  clock,  initially  a  completely  shaded 
circle,  began  to  disappear  when  the  subject  started  to  gather  information 
about  the  alternatives  and  stopped  when  the  subject  selected  an  alternative. 
The  extent  to  which  the  circle  was  no  longer  shaded  was  an  indication  of  the 
amount  of  time  used  by  the  subject  to  decide  among  the  alternatives.  The 
results  from  a  pilot  study  suggested  that  giving  the  clock  200  seconds  to 
fully  disappear  provided  distinguishable  effort  feedback  while  enabling 
subjects  to  decide  among  the  alternatives  before  the  time  expired.  Thus, 
there  was  no  time  constraint :  subjects  had  enough  time  to  easily  finish  each 
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trial.  However,  the  feedback  did  provide  a  potential  indicator  of  the  effort 
expanded. 

Procedure 

Subjects  were  told  that  the  purpose  of  the  experiment  was  to  understand 
how  people  make  decisions  and  that  they  should  select  the  option  they  thought 
best  from  each  set.  Subjects  were  randomly  assigned  to  one  of  the  four 
between- subject  experimental  conditions  defined  by  the  presence  or  absence  of 
accuracy  and  effort  feedback.  Each  subject  was  also  informed  that  if  his  or 
her  performance  exceeded  the  standard  described  above,  he  or  she  would  be 
eligible  for  a  lottery  with  a  prize  of  $100. 

Each  subject  was  run  individually.  First  the  subject  was  instructed  on 
the  Mouselab  system  and  allowed  to  practice  its  use  on  a  series  of  tasks 
unrelated  to  the  present  study.  Subjects  were  then  presented  with  two 
practice  problems  and  the  32  decision  problems.  The  complete  design  was  a  two 
between- subj ect  factors  (accuracy  feedback  absent  or  present,  effort  feedback 
absent  or  present) ,  two  wi thin-subject  factors  (dispersion  low  or  high,  goal 
of  minimizing  effort  or  maximizing  accuracy)  complete  factorial  with  eight 
replicates  for  each  of  the  four  combinations  of  the  within- subj ect  factors. 
Dependent  Measures 

Information  acquisition  and  decision  behavior  can  be  characterized  in 
many  wavs .  can  examine  the  amount  and  sequence  of  information  acquired 

and  the  time  spent  acquiring  information  (Klayman,  1983)  .  We  consider  six 
measures  of  aspt sts  of  decision  processing  to  examine  the  hypotheses.  One 
important  aspect  is  the  total  amount  of  processing.  One  measure  of  amount  is 
the  total  number  of  times  information  boxes  were  opened  for  a  particular 
decision,  denoted  acquisitions  (ACQ) .  A  second  measure  related  to  the  amount 
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of  processing  effort  is  the  total  amount  of  time  spent  examining  information 
in  the  boxes  (BOXTIME) . 

The  next  measures  reflect  the  relative  attention  devoted  to  specific 
types  of  information,  and  hence  are  relevant  to  characterizing  selectivity  in 
processing.  One  measure,  denoted  (PTMI) ,  is  the  proportion  of  the  total  time 
acquiring  information  that  was  spent  in  boxes  involving  the  most  important 
attribute  of  a  particular  decision  problem.  The  attribute  with  the  largest 
weight  was  defined  to  be  the  most  important  attribute. 

The  next  two  measures  are  the  variances  in  the  proportions  of  time  spent 
on  each  alternative  (VAR-ALTER)  and  on  each  attribute  (VAR-ATTRIB) .  Such 
variances  are  related  to  selectivity.  More  compensatory  decision  rules  such 
as  weighted  additive  imply  a  pattern  of  information  acquisition  that  is 
consistent  (low  in  variance)  across  alternatives  and  attributes;  in  contrast, 
noncompensatory  strategies  such  as  EBA  or  lexicographic  imply  more  variance  in 
processing. 

A  final  measure  of  processing  characterizes  the  sequence  of  information 
acquisitions  relating  to  attribute  values.  Given  the  acquisition  of  a 
particular  piece  of  information,  two  particularly  relevant  cases  for  the  next 
piece  of  information  acquired  involve  the  same  alternative  but  different 
attribute  (an  alternative-based,  holistic,  or  a  Type  1  transition),  and  the 
attribute  but  a  different  alternative  (an  attribute-based,  dimensional, 
or  Type  2  transition).  A  simple  measure  of  the  relative  amount  of 
alternative-based  (Type  1)  and  attribute-based  (Type  2)  transitions  is 
provided  by  calculating  the  number  of  Type  1  transitions  minus  the  number  of 
Type  2  transitions  divided  by  the  sum  of  Type  1  and  Type  2  transitions  (Payne, 
1976) .  This  measure  of  the  relative  use  of  alternative-based  versus 
attribute -based  processing,  denoted  PATTERN,  ranges  from  a  value  of  -1.0  to 
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+1.0.  A  more  positive  number  indicates  relatively  more  alternative-based 
processing,  while  a  more  negative  number  indicates  relatively  more  attribute  - 
based  processing. 

In  addition  to  these  six  measures  of  processing,  a  measure  of  relative 
accuracy,  defined  as  above  in  terms  of  the  worst  and  best  choices  for  each 
set,  was  developed  and  denoted  GAIN. 

These  measures  can  be  related  directly  to  the  hypotheses  outlined 
earlier.  The  manipulations  that  should  lead  to  more  normative  patterns  of 
processing,  such  as  the  presence  of  accuracy  feedback  for  low  dispersion 
problems  or  a  goal  emphasizing  accuracy,  for  example,  should  lead  to  higher 
values  of  PATTERN  (more  alternative-based  processing) ;  lower  values  of  VAR- 
ALTER  and  VAR-ATTRIB  (less  selectivity) ;  and  lower  values  of  PTMI  (less  focus 
on  the  most  important  attribute) .  In  addition,  there  should  be  more 
acquisitions  (ACQ)  and  greater  BOXTIME  (more  processing  effort).  Finally, 

GAIN  should  be  higher.  Conditions  leading  to  less  normative  processing  would 
show  the  opposite  pattern  of  results. 

Results 

Overview 

The  main  focus  in  the  results  concerns  how  subjects  adapt  to  the 
manipulations  of  feedback  and  goal.  However,  we  will  also  briefly  present  the 
results  for  dispersion  to  ensure  that  the  prior  results  of  Payne,  Bettman,  and 
Johnson  (1988)  are  replicated.  Effects  are  examined  for  the  four  main  types 
of  dependent  measures  outlined  above:  amount  of  processing,  selectivity  in 
processing,  pattern  of  processing,  and  relative  accuracy. 

Multivariate  Analysis 

Given  the  likely  correlations  among  the  various  dependent  measures,  the 
data  were  first  analyzed  using  a  multivariate  analysis  of  variance  with  two 
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within- subject  factors  (dispersion  and  goal)  and  two  between- sub j ect  factors 
(accuracy  feedback  and  effort  feedback).  The  analysis  included  the  six 
process  measures  described  above  plus  the  measure  of  relative  accuracy  (GAIN) . 
Means  for  these  measures  are  presented  in  Table  1.^ 

There  were  main  effects  of  goal  (F(7,2368)  -  31.8,  p<  .0001)  and 
dispersion  (F(7,2368)  -  32.55,  p<  .0001),  but  no  main  effects  of  accuracy 
feedback  (F(7,71)  -  1.01,  ns)  or  effort  feedback  (F(7,71)  -  .99,  ns).  The 
two-way  interactions  of  accuracy  feedback  by  goal  (F(7,2368)  -  4.52,  p< 

.0001),  effort  feedback  by  goal  (F(7,2368  -  2.46,  p<  .02),  accuracy  feedback 
by  dispersion  (F(7,2368)  =  4.95,  p<  .0001),  and  goal  by  dispersion  (F(7,2368) 
-  3.41,  p<  .0002)  were  significant.  The  other  two-way  interactions,  accuracy 
feedback  by  effort  feedback  (F(7,71)  -  .66,  ns)  and  effort  feedback  by 
dispersion  (F(7,2368)  -  .67,  ns),  were  not  significant.  Finally,  there  was 
one  significant  three-way  interaction  involving  accuracy  feedback,  effort 
feedback,  and  goal  (F(7,2368)  -  2.98,  p<  .004).  The  other  three-way 
interactions  and  the  four-way  interaction  were  not  significant. 

Insert  Table  1  about  here 

Univariate  Analyses 

To  more  completely  characterize  the  effects  of  dispersion,  accuracy 
feedback,  effort  feedback,  and  goal,  separate  univariate  analyses  of  variance 
were  conducted  for  each  of  the  dependent  measures.  The  results  presented  in 
Table  1  will  first  be  discussed  for  dispersion,  then  accuracy  feedback,  then 
effort  feedback,  and  then  for  goal.  Hypothesized  interactions  will  be 
discussed  where  appropriate.  Within  each  section,  the  results  are  presented 
for  the  amount  of  processing  (ACQ  and  BOXTIME) ,  then  for  the  selectivity 
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Table  1 

Summary  of  Process  Measures  and  GAIN  as  a  Function  of  Accuracy  Feedback.  Effort  Feedback. 


Dispersion. _ and  Goal 


Minimize 

Effort 

Goal 

Dispersion: 

Low 

High 

Accuracy  Feedback: 

No 

Yes 

No 

Yes 

Effort  Feedback: 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

ACQ 

17.0 

22.0 

29.1 

24.7 

15.1 

22.2 

25.3 

21.1 

BOXTIME 

8.6 

11.0 

15.8 

12.0 

7.7 

11.4 

12.9 

10.0 

PTMI 

.36 

.39 

.37 

.36 

.43 

.47 

.43 

.  46 

VAR -ALTER 

.09 

.10 

.07 

.09 

.10 

.11 

.08 

.09 

VAR-ATTRIB 

.16 

.13 

.13 

.12 

.21 

.19 

.18 

.18 

PATTERN 

-  .59 

-.52 

-.35 

-  .28 

-.72 

-  .62 

-  .55 

-  .42 

GAIN 

.58 

.61 

.74 

.61 

.74 

.78 

.80 

.72 

Maximize  Accuracy 

Goal 

Dispersion: 

Low 

High 

Accuracy  Feedback: 

No 

Yes 

No 

Yes 

Effort  Feedback: 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

i 

ACQ 

25.8 

27.0 

41.3 

37.8 

23.6 

26.8 

34.7 

1 

31.4  i 

BOXTIME 

15.2 

16.2 

25.9 

20.6 

13.5 

14.2 

20.6 

16.5  | 

PTMI 

.40 

.38 

.32 

.32 

.42 

.42 

.36 

.34 

VAR -ALTER 

.09 

.09 

.06 

.09 

.09 

.08 

.06 

.10 

VAR-ATTRIB 

.15 

.13 

.10 

.09 

.20 

.19 

.12 

.  11 

PATTERN 

-  .49 

-  .42 

-  .19 

-.16 

-  .53 

-  .52 

-  .34 

-  .20 

GAIN 

.61 

.67 

.76 

.75 

.81 

.80 

.78 

.84 
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measures  (PTMI,  VAR-ALTER,  and  VAR-ATTRIB) ,  for  the  PATTERN  measure,  and 
finally  for  relative  accuracy  (GAIN). 

Dispersion .  The  effects  of  dispersion  replicated  those  found  by  Payne, 
Bettman,  and  Johnson  (1988)  for  all  the  variables  except  one.  High  dispersion 
led  to  significantly  less  processing,  more  selectivity,  and  more  attribute- 
based  processing."’  Payne,  Bettman,  and  Johnson  (1988)  found  that  accuracy  was 
not  significantly  different  for  high  dispersion  and  low  dispersion  conditions. 
Thus,  they  concluded  that  the  subjects  in  their  experiment  were  able  to  adapt 
in  ways  that  produced  equivalent  accuracy  with  less  effort  in  the  high 
dispersion  condition.  In  contrast,  the  subjects  in  the  present  study  were 
more  accurate  for  high  dispersion  problems  (M  -  .79  vs.  M  -  .67,  F(l,2518)  - 
89.1,  p  <  .0001).  This  suggests  even  more  strongly  than  the  Payne  et.  al. 
results  that  adaptivity  to  problem  characteristics  can  lend  to  efficient 
decision  processing,  i.e.,  high  accuracy  with  relatively  less  effort. 

Accuracy  Feedback.  The  prediction  for  accuracy  feedback  was  that  such 
feedback  would  have  its  largest  impact  for  problems  with  low  dispersion. 
Accuracy  feedback  was  hypothesized  to  lead  to  more  normative  processing  (i.e.. 
a  greater  amount  of  processing,  less  selectivity,  more  alternative -based 
processing,  and  higher  relative  accuracy).  l4ence,  an  accuracy  feedback  by 
dispersion  interaction  is  predicted.  We  first  present  the  main  effect  results 
for  accuracy  feedback  and  then  consider  the  interactions. 

The  presence  of  accuracy  feedback  increases  the  number  of  acquisitions 
(M  =»  30.1  vs.  M  -  22.6,  F( 1 , 77)  -  4.4,  p<  .04)  and  marginally  increases  the 
time  spent  on  the  information  in  the  boxes  (M  -  16.9  vs.  M  -  12.3,  F(l,77)  = 
3.2,  p  <  .08).  Hence,  more  processing  is  done  with  accuracy  feedback. 

There  is  some  limited  evidence  for  lower  selectivity  under  accuracy 
feedback.  Accuracy  feedback  leads  to  decreased  variance  in  processing  across 
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attributes  (M  -  .13  vs.  M  -  .17,  F(l,77)  -  4.83,  p<  .04).  However,  there  is 
no  significant  effect  of  accuracy  feedback  on  PTMI  (F(l,77)  -  2.24,  ns)  or 
VAR-ALTER  (F(l , 77)  -  1.13,  ns). 

Accuracy  feedback  leads,  as  hypothesized,  to  relatively  more 
alternative-based  processing  (M  -  -.31  vs.  M  -  -.55,  F(l,77)  -  4.4,  p<  .04). 
However,  accuracy  feedback  has  no  main  effect  on  relative  accuracy  ( F ( 1 , 77)  =• 
1.26,  ns) . 

While  these  main  effects  are  roughly  in  the  expected  directions,  our 
hypothesis  was  that  the  main  effects  would  be  qualified  by  an  accuracy 
feedback  by  dispersion  interaction,  with  accuracy  feedback  exhibiting  stronger 
effects  for  the  more  difficult  low  dispersion  problems.  This  hypothesis 
received  support,  as  there  were  accuracy  feedback  by  dispersion  interactions 
for  number  of  acquisitions  (F(l,2518)  -  7.9,  p<  .0001),  time  spent  acquiring 
information  from  the  boxes  (F(l,2518)  -  5.2,  p<  .03),  variance  in  processing 
across  attributes  (F(l,2518)  -  5.0,  p<  .03),  and  relative  accuracy  (F(l,2518) 

-  14.6,  p<  .0001).  As  shown  in  Table  2,  three  of  these  interactions  have  the 
predicted  form:  there  is  more  impact  of  accuracy  feedback  for  low  dispersion 
problems  than  for  high  dispersion  problems  for  ACQ,  BOXTIME,  and  GAIN.  For 
VAR-ATTRIB,  the  low  dispersion,  high  dispersion  condition  shows  the  lowest 
selectivity,  as  predicted,  but  there  is  a  greater  effect  of  accuracy  feedback 
under  high  dispersion.  Only  one  variable,  PATTERN,  displays  a  main  effect  for 
accuracy  feedback  without  the  qualifying  interaction  with  dispersion.  Table  2 
also  shows  that  the  interaction  results  for  PATTERN  have  the  hypothesized 
form,  although  they  do  not  attain  significance. 

Insert  Table  2  about  here 
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Table  2 

Accuracy  Feedback  bv  Dispersion  Interactions 


Dispersion 

Accuracy  Feedback 

No 

Low 

Yes 

High 

No  . 

Yes 

ACQ 

23.0 

33 . 3 

22.1 

28.2 

BOXTIME 

12.8 

18.7 

11.8 

15.1 

VAR-ATTRIB 

.14 

.11 

.20 

.15 

PATTERN 

-  .50 

-.25 

-  .60 

-.38 

GAIN 


.62 


.72 


.78 


.78 
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Thus,  the  results  for  accuracy  feedback  are  generally  as  hypothesized. 
Accuracy  feedback  leads  to  more  normative  forms  of  processing  and  improved 
performance  for  low  dispersion  problems.  Interactions  of  accuracy  feedback 
with  goal  and  with  effort  feedback  and  goal  are  discussed  below. 

Effort  Feedback.  The  effects  of  effort  feedback  can  be  described  fairly 
simply.  There  were  no  significant  main  effects  of  effort  on  any  of  the 
process  measures  or  relative  accuracy.  In  addition,  there  were  no  significant 
accuracy  feedback  by  effort  feedback  interactions.  Importantly,  and  in 
contrast  to  the  results  for  accuracy  feedback,  there  were  also  no  significant 
effort  feedback  by  dispersion  interactions . 

Although  this  lack  of  results  is  as  hypothesized,  one  might  argue  that 
the  manipulation  of  effort  feedback  was  simply  inadequate.  However,  there 
were  interactive  effects  of  effort  feedback  with  other  variables;  in 
particular,  we  discusa  results  below  showing  that’ effort  feedback,  accuracy 
feedback,  and  goal  interact.  Thus,  effort  feedback  appears  to  provide  little 
additional  information  to  subjects  beyond  that  provided  by  process  feedback; 
however,  effort  feedback  may  have  some  effects  on  subjects'  goals. 

Effects  of  Goal.  We  have  argued  above  that  a  goal  (incentive  scheme) 
emphasizing  maximizing  accuracy  will  lead  to  more  normative  strategies  than  a 
goal  emphasizing  minimizing  effort.  In  particular,  we  would  expect  more 
processing,  less  selective  processing,  more  alternative -based  processing,  and 
improved  relative  accuracy  when  the  goal  emphasizes  accuracy  more  than  effort. 
This  pattern  of  results  was  supported,  as  described  below. 

Subjects  did  more  processing  when  the  goal  was  to  maximize  accuracy 
rather  than  to  minimize  effort.  More  information  was  acquired  (M  -  31.4  vs.  M 
=  22.3,  F(l,2518)  -  10.6,  p<  .002)  and  more  time  was  spent  on  the  information 
in  the  boxes  (M  -  18.1  vs.  M  -  11.3,  F(l,2518)  -  159.1,  p<  .0001).  In 


22 

addition,  information  acquisition  was  less  selective  under  a  goal  of 
maximizing  accuracy.  There  was  proportionally  less  time  spent  on  the  most 
important  attribute  (M  -  .37  vs.  M  -  .41,  F(l,2518)  -  30.9,  p<  .0001),  less 
variance  in  processing  over  attributes  (M  -  . 13  vs .  M  -  .17,  F(l,2518)  -  36.9, 
p<  .0001),  and  less  variance  in  processing  across  alternatives  (M  -  .08  vs.  M 
=  .09,  F(l,2513)  -  9.5,  p<  .003).  Finally,  processing  was  more  alternative- 
based  when  the  goal  was  to  maximize  accuracy  (M  -  -.35  vs.  M  =  -.50,  F(l,2466) 
-65.2,  p<  .0001).  Note  that  the  fact  that  there  was  greater  processing  by 
attribute  when  the  goal  was  to  minimize  effort  supports  the  suggestion  of 
Russo  and  Dosher  (1983)  that  attribute -based  processing  is  cognitively  easier. 

Processing  does  become  more  extensive,  less  selective,  and  more 
alternative-based  when  the  goal  is  to  maximize  accuracy,  as  hypothesized. 

Such  processing,  more  consistent  with  normative  strategies,  also  leads  to 
better  performance.  When  the  goal  is  to  maximize  accuracy,  subjects  attain 
greater  relative  accuracy  levels  (M  -  .75  vs.  M  -  .70,  F(l,2518)  -  19.2,  p< 
.0001)  . 

The  fact  that  goal  affects  both  processing  measures  and  accuracy  raises 
the  issue  of  whether  the  processing  changes  mediate  the  effect  of  goal  on 
accuracy.  To  demonstrate  such  mediation,  three  relationships  must  be 
established  (Baron  &  Kenny,  1986):  1)  that  the  goal  manipulation 

significantly  affects  processing;  2)  that  processing  measures  are 
significantly  related  to  accuracy;  and  3)  the  effect  of  goal  on  accuracy  is 
weakened  or  eliminated  if  processing  measures  are  used  as  covariates. 

The  results  reported  above  show  that  goal  significantly  affects  several 
measures  of  processing:  amount  of  information  acquired,  time  spent  on 
information  in  the  boxes,  time  spent  on  the  most  important  attribute,  variance 
in  processing  over  attributes  and  alternatives,  and  the  pattern  of  processing. 
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Hence,  we  next  examined  whether  each  of  these  processing  measures  was 
significantly  related  to  accuracy  over  all  trials  for  all  subjects.^  We 
computed  the  correlation  between  accuracy  and  each  of  these  process  measures. 
The  correlations  were  significant  for  number  of  acquisitions  (r  -  .12,  p  < 
.0001),  time  spent  on  information  in  the  boxes  (r  -  .10,  p  <  .0001),  variance 
in  processing  over  alternatives  (r  -  -.05,  p  <  .01),  and  the  pattern  of 
processing  (r  -  .14,  p  <  .0001).  The  correlations  were  not  significant  for 
variance  in  processing  over  attributes  (r  -  -.02)  or  proportion  of  time  spent 
on  the  most  important  attribute  (r  -  .003).  Hence,  the  first  four  variables 
above  meet  the  second  test  for  mediation,  while  the  latter  two  do  not. 

Finally,  we  examined  the  third  criterion  for  mediation,  the  effect  of 
goal  on  accuracy  with  processing  variables  included  as  covariates.  Since  the 
amount  of  information  acquired  and  the  time  spent  on  information  in  the  boxes 
were  highly  collinear  (r  -  .88),  we  only  used  time  spent  on  information  in  the 
boxes,  variance  in  processing  over  alternatives,  and  pattern  of  processing  as 
covariates.  With  these  covariates  included,  the  effect  of  goal  on  accuracy 
remains  significant  (F(l,2371)  =  8.45,  p  <  .004).  However,  this  effect  is 
weakened,  as  the  F  value  without  the  covariates  was  19.2.  The  three 
covariates  are  all  significant  or  approach  significance  (time  spent  on 
information  in  the  boxes,  F(l,2371)  -  3.53,  p  <  .06;  variance  in  processing 
over  alternatives,  F(l,2371)  -  6.95,  p  <  .01;  and  pattern  of  processing, 
F(l,2371)  »  8.04,  p  <  .005).  Hence,  these  three  variables  meet  the  three 
tests  for  mediation.  However,  since  the  effect  of  goal  on  accuracy  remains 
significant,  the  impact  of  goal  on  accuracy  is  not  completely  mediated  by 
changes  in  the  processing  variables  investigated. 

Interactions  Between  Feedback  and  Goal.  Our  final  hypothesis  stated 
that  accuracy  feedback  and  effort  feedback  will  interact  with  the  decision 


24 

maker's  goal  to  determine  processing  and  performance.  We  argued  that  feedback 
can  also  have  motivational  effects.  When  the  goal  is  to  maximize  accuracy, 
both  accuracy  feedback  and  effort  feedback  may  lead  to  more  normative 
processing  and  better  performance.  When  the  goal  is  to  minimize  effort, 
however,  we  argued  that  effort  feedback  may  lead  to  more  heuristic  processing 
and  reduced  performance  and  that  accuracy  feedback  may  have  little  impact.  We 
examine  these  predictions  initially  by  considering  the  two-way  interactions  of 
accuracy  feedback  with  goal  and  effort  feedback  with  goal.  Then  the  three-way 
interaction  of  accuracy  feedback,  effort  feedback,  and  goal  is  described. 

The  significant  two-way  interactions  of  accuracy  feedback  and  goal  are 
summarized  in  Table  3  for  the  number  of  acquisitions  (F(l,2518)  -  10.6,  p< 
.002),  time  spent  on  the  information  in  the  boxes  (F(l,2518)  =  8.5,  p<  .004), 
proportion  of  time  spent  on  the  most  important  attribute  (F(l,2518)  =22.3,  p< 
.0001),  and  variation  in  processing  across  attributes  (F(l,2518)  -  22.6,  p< 
.0001).  The  interactions  did  not  reach  significance  for  the  other  dependent 
variables . 

Insert  Table  3  about  here 

The  form  of  these  interactions  is  as  predicted.  Note  that  accuracy 
feedback  leads  to  more  normative  processing  to  a  greater  extent  for  a  goal  of 
maximizing  accuracy  than  for  a  goal  of  minimizing  effort.  However,  contrary 
to  the  hypothesis,  there  is  no  interaction  of  accuracy  feedback  and  goal  for 
relative  accuracy.  Performance  is  improved  equally  by  accuracy  feedback 
regardless  of  the  goal . 

The  significant  two-way  interactions  of  effort  feedback  and  goal  are 
presented  in  Table  4  for  time  spent  on  the  information  in  the  boxes  (F(l,2518) 
=  3.0,  p<  .09),  proportion  of  time  spent  on  the  most  important  attribute 
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Table  3 

Accuracy  Feedback  bv  Goal  Interactions 


Goal. 

Accuracy  Feedback 

Minimize 

Effort 

Maximize 

Accuracy 

No 

Yes 

No 

Yes 

ACQ 

19.3 

25.2 

25.9 

36.4 

BOXTIME 

9.8 

12.8 

14.8 

21.0 

PTMI 

.41 

.40 

.40 

.33 

VAR-ATTRIB 

.17 

.15 

.17 

.11 
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(F(l , 2518)  -  5.1,  p<  .03),  and  relative  accuracy  (F(l,2518)  -  4.8,  p<  .03). 
These  interactions  do  not  present  a  clear  picture.  As  hypothesized,  effort 
feedback  leads  to  decreased  relative  accuracy  when  the  goal  is  to  minimize 
effort  and  increased  relative  accuracy  when  the  goal  is  to  maximize  accuracy. 
Also  as  hypothesized,  effort  feedback  leads  to  a  greater  proportion  of  time  on 
the  most  important  attribute  when  the  goal  is  to  minimize  effort  and  a  smaller 
proportion  when  the  goal  is  to  maximize  accuracy.  Contrary  to  the  hypothesis, 
however,  effort  feedback  leads  to  less  time  spent  on  information  in  the  boxes 
when  maximizing  accuracy  is  the  goal  and  has  very  little  impact  on  BOXTIME 
when  the  goal  is  minimizing  effort. 

Insert  Table  4  about  here 

When  the  three-way  interactions  of  accuracy  feedback,  effort  feedback, 
and  goal  are  examined,  a  more  complex  picture  than  that  originally 
hypothesized  emerges.  There  are  significant  three-way  interactions  for  number 
of  acquisitions  (F(l , 2518)  -  3.6,  p<  .06)  and  relative  accuracy  (F91.2518)  = 
8.6,  p<  .004).  These  two  interactions  are  shown  in  Table  5.  WTien  no  accuracy 
feedback  is  available,  effort  feedback  appears  to  increase  acquisitions  and 
relative  accuracy,  regardless  of  goal.  When  accuracy  feedback  is  available, 
however,  effort  feedback  reduces  both  acquisitions  and  performance  when  the 
goal  is  to  minimize  effort.  When  the  goal  is  to  maximize  accuracy,  effort 
feedback  reduces  acquisitions,  but  not  performance,  in  the  presence  of 
accuracy  feedback. 

Insert  Table  5  about  here 
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Table  4 

Effort  Feedback  bv  Goal  Interactions 


Goal 

Effort  Feedback 

Minimize 

Effort 

Maximize 

Accuracy 

No 

Yes 

No 

Yes 

BOXTIME 

11.6 

11.1 

19.3 

16.9 

PTMI 

.40 

.42 

.37 

.36 

GAIN 

.72 

.68 

.74 

.76 
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Table  5 

Accuracy  Feedback  bv  Effort  Feedback  bv  Goal  Interactions 


Goal 

Minimize 

Effort 

Maximize 

Accuracy 

Accuracy  Feedback 

No 

Yes 

No 

Yes 

Effort  Feedback 

No  Yes 

No 

Yes 

No 

.  Yes 

No 

Yes 

ACQ 

16.1  22.1 

27.2 

22.9 

24.7 

26.9 

38.0 

34.6 

GAIN 

.66  .70 

.77 

.67 

.71 

.  73 

.77 

.  79 
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Research  has  shown  that  the  same  individual  will  often  use  a  variety  of 
strategies  to  make  a  decision,  contingent  on  task  demands  (Payne,  1982).  A 
key  question  for  decision  research  today  is  to  better  understand  the  selection 
of  decision  strategies. 

This  paper  has  examined  the  roles  of  effort  and  accuracy  feedback  on 
decision  processes  and  the  role  of  goals  that  emphasize  accuracy  or  effort  on 
strategy  selection.  The  results  show  that  feedback  on  the  accuracy  of 
decisions  leads  to  more  normative -like  processing  of  information  and  improved 
performance.  However,  the  role  of  accuracy  feedback  is  much  greater  for 
certain  types  of  decision  problems.  In  particular,  when  faced  with  decision 
problems  involving  low  dispersion  in  the  weights  of  the  attributes,  accuracy 
feedback  had  a  large  effect  on  decision  processes.  In  contrast,  accuracy 
feedback  had  almost  no  impact  on  the  behavior  of  subjects  faced  with  problems 
involving  high  dispersion  in  weights.  Thus,  accuracy  feedback  appeared  to 
have  its  greatest  impact  when  the  decision  problems  were  more  difficult.  For 
such  problems,  process  feedback  and  general  knowledge  of  what  makes  for  a 
"good"  strategy  are  less  applicable  and  feedback  hence  becomes  more  necessary. 

Explicit  effort  feedback,  on  the  other  hand,  had  no  impact  on  processing 
or  performance  regardless  of  the  dispersion  in  weights.  In  general,  the  idea 
that  people  are  able  to  generate  fairly  adequate  notions  about  their  own 
effort  levels  through  process  feedback  was  supported.  The  only  area  in  which 
effort  feedback  did  appear  to  have  an  impact  was  in  terms  of  interaction  with 
goal.  That  relationship  will  be  discussed  after  we  have  reviewed  the  effects 
of  goals  (incentives)  on  decision  processing. 

The  effort/accuracy  view  of  strategy  selection  implies  that  people 
should  utilize  strategies  that  provide  greater  accuracy  (often  at  the  cost  of 
greater  effort)  when  the  incentives  associated  with  accuracy  are  increased. 
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However,  as  pointed  out  by  several  authors  (e.g.,  Tversky  and  Kahneman,  1986; 
Wright  and  Aboul-Ezz,  1986),  incentives  sometimes  enhance  performance  and  at 
other  times  have  no  effect.  Payne,  Bettman,  and  Johnson  (In  press)  discuss 
these  ambiguous  research  results  in  terms  of  the  common  distinction  between 
working  harder  versus  working  smarter  (see  also  Einhorn  &  Hogarth,  1986; 
Tversky  &  Kahneman,  1986).  That  is,  incentives  may  cause  individuals  to  work 
harder,  but  not  necessarily  smarter. 

Another  contributing  factor  to  the  ambiguity  in  results  may  be 
procedural.  Often  the  procedure  used  in  studies  of  incentive  effects  has  been 
to  simply  increase  the  payoffs  associated  with  good  judgments.  In  the  present 
study,  the  possible  effects  of  incentives  on  the  effort/accuracy  tradeoff 
associated  with  various  strategies  were  more  directly  manipulated  by 
explicitly  emphasizing  either  a  goal  of  maximizing  accuracy  relative  to  effort 
or  minimizing  effort  relative  to  accuracy.  The  results  demonstrated  that  the 
goal  of  maximizing  accuracy  led  to  more  processing  of  information,  less 
selectivity  in  processing,  more  alternative-based  processing  of  information, 
and  improved  performance.  These  results  provide  the  clearest  evidence 
available  to  date  for  the  effects  of  differences  in  goals  on  detailed  process 
tracing  measures  of  decision  strategies  (i.e.,  Billings  and  Scherer,  1988, 
find  relatively  weak  evidence  for  effects  of  decision  importance  on 
processing;  see  also  Ford,  Schemitt,  Schechtman,  Halts  and  Doherty,  1989,  pp . 
101-102,  for  a  similar  appraisal).  To  summarize,  when  the  incentives  were 
structured  to  emphasize  the  goal  of  accuracy  more  than  effort,  we  found  a 
shift  in  strategies  in  the  direction  predicted  by  the  effort/accuracy 
framework . 

As  mentioned  above,  there  were  several  interactions  observed  between  the 
presence  of  effort  feedback  and  goals.  Overall,  it  appeared  that  effort 
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feedback  led  to  decreased  accuracy  when  the  goal  was  to  minimize  effort  and  to 
increased  accuracy  when  the  goal  was  to  maximize  accuracy.  That  is,  effort 
feedback  appeared  to  amplify  the  effects  of  goal.  However,  the  picture  was 
less  clear  when  the  three-way  interactions  of  goal,  effort'  feedback,  and 
accuracy  feedback  were  considered.  In  that  case,  the  overall  pattern  of 
results  departed  from  that  hypothesized  in  several  respects:  1)  Effort 
feedback  only  decreased  acquisitions  and  performance  for  a  minimizing  effort 
goal  when  accuracy  feedback  was  present;  and  2)  Effort  feedback  decreased 
acquisitions  for  a  goal  of  maximizing  accuracy  when  accuracy  feedback  was 
present.  These  patterns  are  intriguing  and  lead  to  some  admittedly 
speculative  interpretations  of  these  discrepant  findings:  a)  When  no  accuracy 
feedback  is  available,  subjects  may  use  effort  feedback  to  help  them  gauge  the 
reasonableness  of  their  decision  processes  (see  Yates  &  Kulik,  1977).  This 
could  hold  even  for  a  minimizing  effort  goal  ’if  subjects  have  some  minimal 
level  of  reasonableness  they  feel  their  decision  processes  should  meet. 

Recall  from  the  description  of  the  performance  index  above  that  accuracy 
receives  some  weight  even  when  the  goal  is  to  minimize  effort.  Hence,  effort 
feedback  may  lead  to  increased  processing  and  performance;  b)  Subjects  may 
achieve  higher  performance  when  they  have  accuracy  feedback  but  no  effort 
feedback  even  for  the  minimize  effort  goal  because  they  can  use  the  accuracy 
feedback  and  are  not  fully  aware  that  they  are  expending  somewhat  more  effort 
in  doing  so.  When  effort  feedback  is  available,  subjects  reduce  their  effort, 
no  longer  rely  on  accuracy  feedback  as  much,  and  their  performance  suffers; 
and  c)  With  both  accuracy  feedback  and  effort  feedback  present,  subjects  may 
learn  to  achieve  the  same  levels  of  performance  with  somewhat  less  effort. 

That  is,  the  presence  of  both  types  of  feedback  may  allow  subjects  to  become 
more  efficient.  Such  efficiency  would  be  most  likely  to  develop  when  the  goal 
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is  to  maximize  accuracy.  When  the  goal  is  to  minimize  effort,  subjects  may 
not  attempt  to  use  accuracy  feedback  as  fully  when  effort  feedback  is  also 
available,  as  noted  above. 

In  summary,  the  present  study  has  shown  that  accuracy  feedback  and 
manipulation  of  the  relative  emphasis  on  maximization  of  accuracy  versus 
minimization  of  effort  affect  strategy  selection  in  ways  consistent  with  a 
general  effort/accuracy  framework.  However,  there  are  complex  interactive 
relationships  between  various  types  of  feedback  and  alternative  goal 
structures  which  remain  unclear.  Further  research  investigating  how 
incentives,  feedback,  and  the  structure  of  decision  tasks  combine  to  affect 
strategy  selection  and  performance  can  make  a  major  contribution  to  our 
understanding  of  effort/accuracy  approaches  to  decision  making. 
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Footnotes 

This  paper  is  based  in  part  on  research  conducted  by  Elizabeth  H.  Creyer  in 
fulfillment  of  the  requirements  for  her  Ph.D.  degree  at  Duke  University. 

four  outcome  gamble  with  a  low  degree  of  dispersion  might  have 
probabilities  of  .30,  .20,  .22,  and  .28  for  the  four  outcomes,  respectively. 

In  contrast,  a  gamble  with  a  high  degree  of  dispersion  might  have 
probabilities  such  as  .68,  .12,  .05,  and  .15  for  the  four  outcomes .  This 
variable  was  included  because  Thorngate  (1980)  had  suggested  the  possibility 
that  probability  information  may  be  relatively  unimportant  in  making  accurate 
risky  choices  (see  also  Beach,  1983). 

9 

In  a  study  with  32  subjects  exposed  to  decision  problems  varying  on 
dispersion,  presence  or  absence  of  dominated  alternatives,  and  incentives, 
subjects  rated  low  dispersion  problems  more  difficult  than  high  dispersion 
choices  (means "of  4.93  vs.  3.91  on  a  scale  where  0  -  not  at  all  difficult  and 

10  -  extremely  difficult,  p<.0001).  In  addition,  the  subjects  stated  that 

they  were  more  confident  about  their  choices  for  low  dispersion  problems  than 

for  high  dispersion  problems  (means  of  7.36  vs.  6 . 54  on  a  scale  where  0  -  not 

at  all  confident  and  10  «  extremely  confident,  p<.0001). 

3 

In  order  to  demonstrate  that  subjects  with  effort  feedback  did  not  perceive 
greater  time  pressure,  after  they  completed  all  32  choices  subjects  were  asked 
to  indicate  on  a  1  (low)  to  7  (high)  scale  how  much  time  pressure  they 
experienced.  There  were  no  significant  effects  of  effort  feedback  or  accuracy 
feedback  on  this  measure . 

^  These  means,  and  the  univariate  analyses  of  variance  reported  below,  are 
based  upon  the  total  number  of  cases  for  each  variable.  The  multivariate 
analysis  of  variance  eliminates  any  observation  missing  any  variable,  so  there 
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are  fewer  cases  for  the  multivariate  analysis.  The  means  are  virtually 
identical  for  the  two  cases . 

^  High  dispersion  led  to  fewer  acquisitions  (ACQ)  (M  -  25.3  vs.  M  -  28.4, 

F ( 1 , 2518)  -  20.2,  p  <  .0001),  less  BOXTIME  (M  -  13.5  vs.  M  -  15.9,  F(l,2518)  - 
19.5,  p  <  .0001),  more  focus  on  the  most  important  dimension  (PTMI)  (M  -  .42 
vs.  M  -  .36,  F(l,2518)  -  60.0,  P  <  .0001),  higher  selectivity  for  attributes 
(VAR-ATTRIB)  (M  -  .17  vs.  M  -  .12,  F(l,  2518)  -  94.0,  p  <  .0001),  and  more 
attribute-based  processing  (lower  values  of  PATTERN)  (M  -  -.48  vs.  M  -  -.37, 
F(l,2466)  -  36.6,  p  <  .0001).  The  effect  of  dispersion  on  variance  in 
processing  across  alternatives  (VAR-ALTER)  was  not  significant  (F(l,2513)  =* 
2.65,  ns) . 

^  The  number  of  trials  available  for  these  correlations  ranged  from  2467  to 
2519,  depending  upon  the  number  of  cases  with  missing  data  for  each  variable. 
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